Automatic Continuous Speech Recognition with Rapid Speaker Adaptation for Human/machine Interaction

This thesis presents work in three main directions of the automatic speech recognition field. The work within two of these – dynamic decoding and hybrid HMM/ANN speech recognition – has resulted in a real-time speech recognition system, currently in use in the human/machine dialogue demonstration system WAXHOLM, developed at the department. The third direction is fast unsupervised speaker adaptation, where “fast” refers to adaptation with a small amount of adaptation speech. The work in dynamic decoding has involved the development of a continuous speech decoding engine based on the A* search paradigm. An efficient implementation of the algorithms has made real-time continuous speech recognition possible in the WAXHOLM dialogue system with a lexicon of about 1000 words. Features of the search algorithms that are important for the real-time performance are proposed. These include efficient use of beam-pruning, and graph reduction methods that greatly reduce the effective search space. The hybrid HMM/ANN recognition is an area of work in its own right, but is also important in the speaker adaptation experiments. A very flexible ANN architecture has been developed and refined during the course of the thesis work. The architecture is a generalization of the TDNN and the RNN architecture, and allows both delayed and look-ahead connections. In the latest experiments, sparsely connected networks were investigated. Sparsely connected networks were shown to perform significantly better than their fully connected counterparts with an equal number of connections. In an experiment with phoneme recognition of the TIMIT database, the recognition rate of the hybrid HMM/ANN system is in the range of the highest reported, and only outperformed by another hybrid system. The fast speaker adaptation work is based on the notion that an explicit a priori model of the speaker variability helps to rapidly adapt to a new speaker. In the experiments, a parametric speaker characterization is introduced in the ANN by adding special-purpose speaker-space input units whose activity values are determined by the speaker adaptation. Experiments have been made both with the American English TIMIT database and the Swedish WAXHOLM database, and a positive adaptation effect is detected after only a few syllables.